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The  explicit  incorporation  of  measurement  and  analysis  as  a  distinct  process  area  in  the  Capability  Maturity  Modd 
Integration  (CMMP)  provides  management  with  the  visibility  and  focus  that  organizations  need  to  guide  the  use  of  meas¬ 
urement  in  thdr  process  improvement  efforts,  which  was  missing  in  previous  modds.  This  artide  reviews  the  content  and 
rationale  bdiind  the  new  process  area  and  describes  how  the  ideas  introduced  there  are  further  daborated  and  evolved  through¬ 
out  capability  maturity  modd  integration  modds. 


Measurement,  or  the  need  for  it,  is  per¬ 
vasive  in  software  and  systems  engi¬ 
neering.  Yet  an  understanding  of  how  to 
best  use  measurement  has  remained  all 
too  uncommon,  as  has  straightforward 
guidance  from  the  experts. 

Even  experts  have  difficulty  following 
the  sometimes- implicit  threads  among  the 
measurement-related  concepts  in  many 
process  improvement  models  and  stan¬ 
dards.  Exhortations  to  do  measurement, 
without  sufficiently  elaborating  what  to 
do,  have  not  worked  exceptiondly  well  in 
the  past.  However,  there  has  been  an 
increasing  recognition  of  the  importance 
of  focusing  explicitly  on  measurement 
and  analysis.  Certain  basic  ideas  must  be 
introduced  early  and  well. 

The  Need  for  an  Early  Focus 

The  need  to  focus  on  measurement  and 
analysis  from  the  beginning  of  a  process 
improvement  effort  has  not  always  been 
well  understood,  even  by  experienced 
assessors  and  expert  consultants.  Yet 
organizations  that  have  succeeded  in  put¬ 
ting  successful  measurement  programs  in 
place  often  say  they  could  have  avoided 
much  grief  and  struggle  with  rework  had 
they  focused  on  how  to  implement  meas¬ 
urement  and  analysis  correctly  in  the  earli¬ 
er  phases  of  their  process  improvement 
efforts. 

Indeed,  some  organizations  have  cre¬ 
ated  their  own  measurement  process  areas 
to  guide  their  improvement  efforts, 
become  more  competitive,  and  enhance 
their  ability  to  more  quickly  achieve  a 
higher  level  of  process  maturity.  In  fact, 
measurement  is  so  important  to  the  suc¬ 
cess  of  software  projects  that  the  U.S. 
D  epartment  of  D  efense  requires  the  fol¬ 
lowing  of  all  major  programs: 

...  to  have  a  software  measure¬ 
ment  process  to  plan  and  track  the 
software  program  and  to  assess 
and  improve  the  development 


process  and  associated  software 
product.  [1] 

Focusing  on  measurement  can  provide 
much  value  to  both  projects  and  organiza¬ 
tions.  Measurement  enables  project  man¬ 
agers  to  answer  the  following  questions: 

•  Is  there  really  a  problem? 

•  How  big  is  the  problem? 

•  What  is  the  scope  of  the  problem? 

"M  easurement  helps 
provide  objective  insight 
into  issues  and 
processes,  along  with  the 
ability  to  objectively 
identify  and  manage 
risks  and  provide  early 
detection  and  resolution 
of  problems. " 


•  What  is  causing  the  problem? 

•  Are  there  relat^  problems? 

•  Can  I  trust  the  data? 

•  What  should  I  expect;  what  will  hap¬ 
pen? 

•  What  are  my  alternatives? 

•  What  is  the  recommended  course  of 
action? 

•  When  can  I  expect  to  see  results? 
Measurement  helps  provide  objective 

insight  into  issues  and  processes,  along 
with  the  ability  to  objectively  identify  and 
manage  risks  and  provide  early  detection 
and  resolution  of  problems.  Measurement 
also  facilitates  evidence-based  team  com¬ 
munication  and  enables  objective  planning 
and  estimating  and  the  ^hity  to  assess 
organizational  performance  in  an  unbi¬ 


ased  and  defensible  manner.  These  in  turn 
provide  an  objective  basis  for  defending 
and  justifying  decisions.  Finally,  measure¬ 
ment  provides  information  that  improves 
decision  making  in  time  to  affect  the  busi¬ 
ness  or  mission  outcome  [2]. 

Doing  Measurement  Ri^ 

0  f  course,  many  approaches  to  software 
measurement  now  exist.  Several  published 
international  standards  address  software 
measurement  and  closely  related  issues. 
There  also  exists  voluminous  literature  in 
software  measurement  and  metrics,  and  a 
rich  body  of  literature  in  statistics  and 
quantitative  methods  dating  back  well  over 
a  century. 

Negotiating  the  morass  of  standards, 
models,  guidebooks,  courses,  and  expert 
consultants  can  be  a  daunting  task. 
Fortunately,  though,  there  is  a  clearly 
emerging  community  of  practice  that 
spans  both  software  measurement  and 
process  improvement.  In  fact,  the 
Capability  Maturity  Model  Integration 
(CMMI®)  Measurement  and  Analysis 
process  area  was  developed  in  a  collabora¬ 
tive,  coordinated  fashion  with  colleagues 
who  also  worked  concurrently  with  the 
Practical  Software  and  Systems 
Measurement  Support  Center,  and  worked 
on  the  development  of  the  emerging  ISO 
[International  Organization  for  Standard¬ 
ization]  standards  on  both  software  meas¬ 
urement  and  process  assessment.  People 
working  in  the  field  are  also  closely  cou¬ 
pled  with  related  work  in  standards  and 
with  groups  such  as  the  International 
Function  Point  Users  Group  (IFPUG)l 

Measurement  andAnalysis  in 
Capability  Maturity  Model 
Inte^'ation  Models 

The  Measurement  and  Analysis  process 
area  is  an  important  addition  to  the 
CMMI.  Its  scope  is  much  wider  and  more 
explicit  than  the  treatment  of  measure- 
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ment  in  the  Capability  Maturity  Model  for 
Software  (SW-CMM)  [3].  The  SW-CMM 
contains  a  measurement  and  analysis  com¬ 
mon  feature,  the  practices  of  which  apply 
to  the  institutionalization  of  the  model's 
key  process  areas.  Akin  to  generic  prac¬ 
tices  in  capability  maturity  model  integra¬ 
tion  models,  these  practices  are  meant  to 
control  and  improve  the  performance  of 
the  processes  themselves.  Some  measure¬ 
ment-related  practices  also  exist  in  various 
places  in  the  activities-performed  com¬ 
mon  feature  in  the  SW-CMM,  but  a  single, 
coherent  treatment  does  not  exist  for  what 
is  reguired  to  establish  and  sustain  a  viable 
measurement  and  analysis  process. 

"The  purpose  of  measurement  and 
analysis  is  to  develop  and  sustain  a  meas¬ 
urement  capability  that  is  used  to  support 
management  information  needs"  [4].  The 
Measurement  and  Analysis  process  area 
supports  all  process  areas  by  providing 
practices  that  guide  projects  and  organiza¬ 
tions  in  aligning  their  measurement  needs 
and  objectives  with  a  measurement 
approach  that  will  provide  objective 
results  that  can  be  used  in  making 
informed  decisions  and  by  taking  appro¬ 
priate  corrective  actions. 

As  discussed  more  fully  in  the  process 
area  itself,  measurement  and  analysis  prac¬ 
tices  are  organized  under  two  specific 
goals  that  are  aimed  at  (1)  aligning  meas¬ 
urement  activities  with  identified  informa¬ 
tion  needs  and  objectives,  and  (2)  provid¬ 
ing  data  analyses  and  results  that  address 
those  needs  and  objectives.  These  goals 
may  be  achieved  by  the  successful  per¬ 
formance  of  their  respective  specific  prac¬ 
tices  shown  in  Figure  I*. 

The  specific  practices  associated  with 
the  first  goal  est^lish  a  coherent  plan  for 
measurement  and  analysis.  They  address 
these  guestions:  "Why  are  we  measuring?" 
"What  are  we  going  to  measure?"  "How 
are  we  going  to  measure?"  and  "What  will 
be  done  with  the  data  once  we  have 
them?"  The  specific  practices  associated 
with  the  second  goal  advise  the  user  to 
just  do  it.  Of  course,  the  ultimate  goal  is 
as  follows: 

...  to  get  the  results  of  performing 
measurement  and  analysis  into  the 
hands  of  those  who  will  take 
action  based  on  the  results.  The 
process  area  emphasizes  the  need 
that  results  must  be  communicat¬ 
ed  to  those  needing  the  informa¬ 
tion.  [5] 

0  ther  guidance  about  what  constitutes 
good  measurement  practices  does  exist  in 
capability  maturity  model  integration 


models,  most  notably  in  some  of  the 
process  areas  with  a  legacy  in  the  source 
documents  on  which  the  CMMI  is  based. 
Those  process  areas  do  contain  certain 
specific  practices  that  reguire  measure¬ 
ment  activities  to  be  performed.  As  seen 
more  fully  in  the  CMMI,  the  Measurement 
and  Analysis  process  area  makes  explicit 
reference  to  other  process  areas  -  in  par¬ 
ticular,  Organizational  Process  Definition 
and  the  heavily  measurement-oriented 
process  areas  at  CMMI  Levels  4  and  5. 
With  the  addition  of  the  Measurement 
and  Analysis  process  area,  the  CMMI 
summarizes  much  of  the  experience  base 
on  which  the  proper  conduct  of  measure¬ 
ment  and  analysis  relies. 

Maturing  Measurement 
Capability 

The  Measurement  and  Analysis  process 
area  provides  a  central  focus  that  describes 
good  measurement  practice.  But  the 
process  area  does  not  stand  alone.  The 


CMMI  also  provides  important  guidance  in 
its  generic  goals  and  practices,  some  of 
which  have  explicit  measurement  content. 
The  generic  practices  serve  together  to 
help  institutionalize  measurement  and 
andysis,  or  any  other  process,  and  to 
improve  the  capability  with  which  measure¬ 
ment  and  analysis  are  performed  over  the 
life  cycle  of  the  product  and  organization. 

Like  any  other  process  area,  measure¬ 
ment  and  analysis  can  progress  from  being 
performed  in  an  essentially  ad  hoc  man¬ 
ner,  through  following  a  well-defined 
measurement  process,  to  using  measure¬ 
ment  to  evaluate  and  improve  the  meas¬ 
urement  process  itself.  Several  CMMI 
generic  practices  have  a  clear  measure¬ 
ment  flavor  (Table  1).  However,  all  of  the 
generic  practices  can  be  applied  to  the 
conduct  of  measurement  and  analysis®. 

Several  generic  practices  discuss  orga¬ 
nizational  pohcies,  sufficiency  of  resourc¬ 
es,  explicit  assignment  of  responsibilities, 
and  training  provisions.  These  help  estab- 


Table  1:  M easurement-R elated  G  eneric Practices  [4] 


Practice 

Statement 

2.8  Monitor  and  Control 
the  Process 

Monitor  and  control  the  process  against  the  plan  for  performing  the  process  and 
take  appropriate  corrective  action. 

3.2  Collect  Improvement 
Information 

Collect  work  products,  measures,  measurement  results,  and  improvement  information 
derived  from  planning  and  performing  the  process  to  support  the  future  use  and 
improvement  of  the  organization's  processes  and  process  assets. 

4.1  Establish  Quantitative 
Obiectives  for  the  Process 

Establish  and  maintain  quantitative  objectives  for  the  process  about  quality  and  process 
performance  based  on  customer  needs  and  business  objectives. 

4.2  Stabilize  Sub-Process 
Performance 

Stabilize  the  performance  of  one  or  more  sub-processes  of  the  process  to  determine  its 
ability  to  achieve  the  established  quantitative  quality  and  process  performance 
objectives. 

5.1  Ensure  Continuous 

Process  Improvement 

Ensure  continuous  improvement  of  the  process  in  fulfilling  the  relevant  business  goals 
of  the  organization. 

5.2  Correct  Root  Causes 
of  Problems 

Identify  and  correct  the  root  causes  of  defects  and  other  problems  in  the  process. 
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lish  and  sustain  process  capability  and  a 
commitment  to  doing  measurement  reg¬ 
ularly  and  well.  Indeed,  they  provide  the 
organizational  infrastructure  that  is  nec¬ 
essary  to  implement  and  institutionahze 
any  process. 

Other  generic  practices  provide  guid¬ 
ance  for  planning  and  related  activities. 
The  planning-related  generic  practices, 
including  the  estabhshment  of  guantita- 
tive  objectives  (4.1)  and  improvement 
objectives  (5.1)  help  establish  the  scope 
and  objectives  for  measurement  work. 
Although  they  relate  to  both  specific 
goals  of  the  Measurement  and  Analysis 
process  area,  they  are  particularly  impor¬ 
tant  for  the  alignment  activities  dis¬ 
cussed  in  specific  goal  1  hsted  earher. 

Several  generic  practices,  including 
stabihze  sub-process  performance  (4.2), 
guide  the  performance  and  management 
of  any  process.  These  also  support  both 
specific  goals  1  and  2  of  Measurement 
and  Analysis,  but  are  particularly  impor¬ 
tant  for  doing  measurement  and  analysis 
and  reporting  the  results. 

Finally,  along  with  others,  the  meas¬ 
urement-oriented  generic  practices  that 
reguire  monitoring  and  controlling  the 
process  (2.8),  collecting  improvement 
information  (3.2),  and  correcting  root 
causes  of  problems  (5.2)  help  evaluate 
and  improve  the  conduct  of  the  meas¬ 
urement  process  itself.  Together  they 
provide  an  evidential  basis  for  improving 
the  manner  in  which  future  measure¬ 
ment  and  analysis  are  done’. 

Implementing  Good 

Measurement  Practice 

Along  with  the  measurement- related 
generic  practices,  the  Measurement  and 
Analysis  process  area  provides  essential 
guidance  about  what  to  do  whenever 
tirere  is  a  need  for  measurement.  How¬ 
ever,  measurement  and  analysis  are 
always  done  in  the  context  of  perform¬ 
ing  other  processes. 

The  Measurement  and  Analysis 
process  area  seamlessly  integrates  meas¬ 
urement  activities  with  those  other 
processes  to  address  a  wide  variety  of 
both  project  and  organization-wide 
information  needs  and  provides  a  basis 
for  integrating  measurement  and  analysis 
with  process  definition. 

L&e  other  support  functions,  the 
Measurement  and  Analysis  process  area 
serves  multiple  purposes.  Every  process 
area  is  dependent  to  some  extent  on 
properly  using  measurement  and  analy¬ 
sis.  The  engineering  and  management 
process  areas  describe  the  sources  of  the 


contractual  reguirements,  other  informa¬ 
tion  needs,  and  business  objectives  with 
which  the  measurement  and  analysis 
activities  are  aligned.  In  turn,  the  results 
of  the  measurement  activities  are  pro¬ 
vided  back  to  inform  the  work  described 
by  those  same  process  areas. 

Maturing  Analytic  Capability 

Measurement  is  applied  differently  as  the 
organization  successfully  satisfies  the 
goals  of  more  and  more  CMMI  process 
areas.  It  typically  begins  with  a  focus  on 
clarifying  sometimes-implicit  business 
objectives  and  informational  needs  and 
translating  them  into  measurable  objec¬ 
tives.  A  basic  set  of  skills,  resources,  and 

"M  oreover,  relying  on 
a  well-defined 
measurement  process 
can  demonstrate 
evidence  of  business 
value,  thus  helping  justify 
continued  Investment  in 
process  improvements 
and  the  measurement 
and  analysis  activities 
that  support  it." 

experiences  is  built  for  the  future. 
Measurement  often  starts  with  using 
simple  charts  and  graphs,  but  as  the 
organization  matures,  demand  increases 
for  more  sophisticated  guantitative 
analyses  such  as  statistical  process  con¬ 
trol  (SPC),  structural  modeling,  or  other 
multivariate  statistical  methods. 

As  the  organization's  analytic  sophis¬ 
tication  increases,  finer- grained  meas¬ 
ures  of  defects  and  product  guality  are 
coupled  exphcitly  widi  process  perform¬ 
ance.  By  the  time  the  organization  reach¬ 
es  CMMI  Level  4,  routine  reliance  on 
guantitative  management  enhances 
process  discipline.  After  Level  5  is 
attained,  there  is  increased  use  of  work- 
product  inspections,  systematic  pro¬ 
grams  of  defect  prevention  driven  by 
causal  analysis,  and  improved  process 
performance  that  often  leads  to  marked¬ 
ly  increased  predictabihty  of  productivi¬ 
ty  and  schedule. 


Analytic  Approaches 

Many  statistic^  analytic  solutions  to  meas¬ 
urement  problems  are  possible  in  software 
process  improvement;  however,  few  are 
widely  used.  For  example,  in  an  unpub¬ 
lished  survey  of  representatives  of  high- 
maturity  organizations,  almost  90  percent 
reported  that  SPC  control-charting  tech- 
nigues  were  in  common  or  standardized 
use  in  their  organizations  [6].  Only  Pareto 
analyses®  appear  to  be  more  common. 

Such  wide  reliance  on  SPC  and  Pareto 
analyses  is  due  in  large  part  to  the  SW- 
CMM  and  its  heritage  in  industrial  engi¬ 
neering  and  total  guality  management,  and 
also  because  graphical  presentations  are 
intuitive®.  Of  course,  not  all  problems 
have  the  same  solution.  SPC,  for  example, 
is  only  one  tool,  and  it  is  not  always  used 
correctly.  0  ne  size  does  not  necessarily  fit 
all,  yet  there  still  is  relatively  little  evidence 
supporting  use  of  alternative  data  analytic 
approaches. 

Although  the  use  of  designed  experi¬ 
ments  and  guasi- experimentation  fits 
guite  naturally  into  applications  of  causal 
analysis  and  defect  prevention  [7,  8],  they 
still  are  not  widely  used  in  higher-maturity 
organizations;  fewer  than  10  percent 
reported  that  they  used  designed  experi¬ 
ments,  and  only  two  respondents  said  they 
used  guasi- experimental  designs  [6]. 

There  is  evidence,  however,  that 
experimental  methods  may  be  used  more 
commonly  than  you  might  think.  For 
example,  in  a  recent  study  of  practitioners 
and  users  of  software  measurement, 
almost  40  percent  said  their  organizations 
commonly  employ  experiments  and/or 
pilot  studies  prior  to  the  widespread 
deployment  of  major  additions  or 
changes  to  development  processes  and 
technologies.  Undoubtedly,  not  all  of 
them  follow  rigorous  methodological 
standards.  However,  the  results  are 
encouraging  because  the  study  sample  was 
structured  to  include  representatives  from 
organizations  that  have  had  varying  suc¬ 
cess  with  their  software  measurement 
efforts;  there  were  fahures  as  well  as  suc¬ 
cesses  [9]. 

One  occasionally  sees  more  sophisti¬ 
cated  uses  of  curve  fitting,  for  example, 
using  Rayleigh  curve-based  models’".  Six 
Sigma  approaches  also  are  gaining 
increased  interest  in  the  process  improve¬ 
ment  community  [10, 11].  However,  wide¬ 
spread  use  remains  uncommon  [6]. 
Indeed,  there  still  is  minimal  use  of  multi¬ 
variate  methods,  classical  or  otherwise. 
Even  basic  statistical  methods  such  as 
regression  analysis  or  analysis  of  variance 
are  reportedly  used  by  relatively  few  high 
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maturity  organizations  [6,  9]. 

Available  Guidance 

With  the  exception  of  SPC,  capability 
maturity  model  integration  models  do  not 
offer  a  great  deal  of  guidance  on  using 
data  analytic  methods.  Indeed,  even 
ISO/IEC  15939  [12]  focuses  more  on 
measurement  fundamentals  as  opposed  to 
detailed  guidance  about  analysis.  With  the 
notable  exception  of  the  Practical 
Software  and  Systems  Measurement 
Support  Center's  work  and  their  Practical 
Software  Measurement  Insight  tool",  the 
same  is  true  for  all  of  the  best-known 
frameworks,  models,  and  standards  in  the 
process  improvement  community.  Yet  any 
raw  data  must  be  analyzed  and  interpreted 
with  care. 

The  early  emphasis  is  on  presentation 
graphics  in  most  software  measurement 
programs,  probably  because  visual  dis¬ 
plays  often  appear  to  be  intuitive.  But  they 
also  are  often  misused  and  misinterpreted 
[13,  14].  The  challenge  to  the  measure¬ 
ment  community  is  to  balance  rigor  and 
methodological  defensibility  with  clarity 
and  practice  import.  There  is  often  a  very 
real  culture  clash  between  measurement 
experts  who  are  trained  to  attend  to  excru¬ 
ciatingly  obtuse  detail  and  practitioners 
who  need  actionable  guidance. 

Classic  tools  for  process  improvement 
in  the  manufacturing  world  date  back  at 
least  to  Doming  [15]  and  Juran  [16]. 
Techniques  such  as  Pareto  charts,  run 
charts,  histograms,  pie  charts,  scatter  dia¬ 
grams,  bar  graphs,  and  control  charts  are 
first  principles  for  any  good  manager  or 
practitioner  in  most  engineering  disci¬ 
plines.  These  are  not  advanced  topics  by 
any  means,  although  their  proper  applica¬ 
tion  and  interpretation  do  t^e  trying 
and  experience.  Indeed,  control  charts  are 
often  posted  on  the  walls  for  easy  refer¬ 
ence  in  many  enterprises,  but  not  often  in 
software  organizations,  which  is  some¬ 
thing  of  an  anomaly.  The  software 
process  improvement  community  often 
seems  to  have  forgotten  its  heritage  in 
total  quality  management  and  industrial 
engineering. 

Detailed,  prescriptive,  how-to  guid¬ 
ance  is  outside  the  province  of  capability 
maturity  model  integration  models.  But 
other  sources  of  guidance  do,  of  course, 
exist.  Many  other  books  and  articles  exist 
in  the  published  literature  on  both  applied 
statistics  and  software  measurement". 
And  many  courses  for  measurement  prac¬ 
titioners  dso  are  available. 

Summary  and  Conclusions 

A  successful  process  for  measurement 


and  analysis  is  characterized  by  decision 
making  that  regularly  includes  data  analy¬ 
ses  results  that  are  based  on  objective 
measurement.  Following  such  a  process 
can  help  projects  and  organizations  make 
significant  performance  improvements  in 
their  other  software  processes  and  in  the 
products  and  services  that  those  process¬ 
es  help  bring  about.  Moreover,  relying  on 
a  well-defined  measurement  process  can 
demonstrate  evidence  of  business  value, 
thus  helping  justify  continued  investment 
in  process  improvement  and  the  measure¬ 
ment  and  analysis  activities  that  support  it. 

The  sophisticated  use  of  measurement 
and  analysis  that  characterizes  high-matu¬ 
rity  organizations  has  been  shown  to 
result  in  substantial  added  value  [17]. 
Organizations  that  have  attained  high- 
maturity  status  regularly  report  notable 
improvements  in  measured  customer  sat¬ 
isfaction  as  well  as  better  schedule  and 
budget  predictability.  These  businesses 
also  commonly  provide  evidence  of 
increased  staff  productivity,  as  measured, 
for  example,  by  reductions  in  develop¬ 
ment  effort  per  line  of  code  or  function 
point.  Also  they  demonstrate  heightened 
product  quality  with  earlier  defect  detec¬ 
tion  profiles  and  marked  decreases  in 
defect  density  during  testing. 

Measurement  is  a  key  enabler  for 
process  improvement  and  enhanced  prod¬ 
uct  quality.  An  organization  with  a  mature 
approach  to  measurement  and  analysis 
vrill  have  confidence  in  its  abilities  to 
effectively  deliver  products  that  meet  its 
customer's  needs.  Measurement  must 
begin  early  if  it  is  to  reach  its  full  poten¬ 
tial,  and  measurement  capability  must 
grow  over  time.  It  is  difficult  for  us  to 
conceive  of  serious  software  engineering 
or  accomplished  management  without 
measurement.  The  time  has  come  for 
software  and  systems  development  to 
move  toward  using  the  same  degree  of 
sophistication  in  measurement  that  other 
engineering  disciplines  have  used  for 
many  decades.4 
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Notes 

1.  This  text  is  an  abridgment  and  exten¬ 
sion  of  material  that  previously 
appeared  in  the  following:  Inter¬ 
national  Eunction  Point  Users  Group. 
IT  Measurement:  Practical  Advice 

from  the  Experts.  Eds.  D.  Herron,  J. 
Russac,  D.  Coley,  J.  Curfman,  B. 
Emmons,  andj.  Schofield.  New  York: 
Addison-Wesley  17  Apr.  2002:  577- 
604. 

2.  We  called  it  the  "Measurement  Mafia" 
when  we  were  creating  the  Measure¬ 
ment  and  Analysis  Process  Area  for 
CMMI. 

3.  See,  for  example,  Emmons,  B.,  and  C. 
Dekkers.  "Function  Point  (FP) 
Maturity  Model:  How  FP  Supports  the 
Capability  Maturity  Integration 


(CMMI)  Model."  CrossTal  k  Feb. 
2002. 

4.  The  data  flow  arrows  in  the  figure  are 
modified  from  the  original. 

5.  This  figure  is  based  on  a  similar  one  in 
the  CMMI  Training  Materials,  which 
are  not  publicly  available. 

6.  The  higher  capability  level  generic 
practices  are  not  included  in  the  staged 
version  of  the  V1.02  CMMI  models. 

7.  In  fact,  measurement  experts  routinely 
subject  their  own  work  to  empirical 
evaluation. 

8.  For  an  introduction  to  Pareto  analysis, 
see  Florae,  W.,  and  A.  Carleton. 
Measuring  the  Software  Process: 

Statistical  Process  Control  for  Software 

Process  Improvement.  Reading,  MA: 
Addison-Wesley,  1999:  62-64 

9.  Ibid.  This  is  a  good  source  for  a  use¬ 
ful  treatment  of  SPC  in  software 
process  improvement. 

10.  A  Rayleigh  curve  yields  a  good  approx¬ 
imation  to  the  actual  labor  curves  on 
software  projects. 

11.  The  PSM  treatment  of  data  analysis 
focuses  on  graphical  presentations. 
Their  examples  range  from  the  very 
simple  to  sometimes  guite  sophisticat¬ 
ed  uses  of  multivariate  graphical  indi¬ 


cators.  See  McGany  J.,  D.  Card,  C. 
Jones,  B.  Layman,  E.  Bailey,  J.  Dean, 
and  F.  Hall.  Practical  Software  and 
Systems  Measurement:  A  Foundation 

for  Objective  Project  Management. 

V4.0b.  Department  of  Defense  and 
U.S.  Army,  Oct.  2000,  particularly  part 
4,  chapter  3. 

12.  A  few  good  places  to  start  include  Van 
Solingen,  R.,  and  E.  Berghout.  The 
Goal/ Question/ Metric  Method.  New 
York:  McGraw-Hill,  1999;  Fenton,  N. 
E.,  and  S.  L.  Pfleeger.  Software 
Metrics:  A  Rigorous  and  Practical 

Approach.  2nd  ed.  Boston,  MA:  PWS 
Publishing  Company,  1999;  and  Florae, 
W,  and  A.  Carleton.  Measuring  the 
Software  Process:  Statistical  Process 

Control  for  Software  Process 

Improvement.  Reading,  MA:  Addison- 
Wesley,  1999;  as  well  as  the  work  by 
Tufte  [13],  Wohlin  [7],  and  Card  [10] 
cited  earlier.  Journal  articles  with  exam¬ 
ples  of  data  analytic  methods  worth 
emulating  often  can  be  found  in 
Empirical  Software  Engineering:  An 

International  Journal  ["International 
Symposium  on  Empirical  Software 
Engineering."  IEEE  Computer  Society 
Staff,  2002]. 
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